Image Based Rendering in Computer Graphics

The fields of computer vision and computer graphics are, in some sense, the duals of each other. In computer vision the goal is to extract information about a 3D scene from a single image or many images of the scene taken from different viewpoints, while the main goal of a computer graphics system is to generate 2D images (or a sequence of 2D images in the case of computer animation) from 3D data about the scene. The process of actually generating a 2D picture or image from 3D data is known as rendering. Since the 3D scene is eventually displayed on a 2D monitor or screen, the role of rendering in the overall graphics pipeline is a fundamental one. Traditionally, a description of the 3D scene being rendered is provided by a detailed and complex model of the scene. Image based rendering (IBR) is a relatively recent and evolving technique in which the image of a scene from a particular viewpoint is generated not from a model, but from a set of images taken from different viewpoints [4 ]. This is closely related to the problem of structure from motion in computer vision, where the goal is to obtain 3D shape information about an object from multiple views of the object. Thus, it is not surprising that the techniques used to perform image based rendering are similar to those found in computer vision and specifically those used to solve the structure from motion problem. In order to better understand the techniques used in IBR, a brief description of the structure from motion problem in computer vision along with an overview of rendering in computer graphics is provided next.